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Overview 


• Review Traffic Scenarios 

• Overview of Well-Clear ACES Simulations 

• Review of Well-Clear Definitions within Autoresolver 

• Lessons Learned 

• Results 
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But First... 


"Parallelisms of weak-side defense in basketball and 

UAS DAA systems" 


• Flight hours 

• Big sky theory 

• Lack of intruder intent 

• Maneuvering trajectories 

• Limited field of regard 

• Aircraft performance 
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UAS Missions 


Atmospheric Sampling 

- Global Hawk (RQ4-A) [2350 flights] 

Border Patrol 

- Global Hawk (RQ4-A) [665 flights] 

Cargo Transport 

• UAS Cessna 208 [1320 flights] 

Strategic Wildfire Monitoring 

- Predator B (MQ-9) [325 flights] 

Air Quality Monitoring 

- Shadow-B (RQ7B) [1050 flights] 

On-Demand Air Taxi 

- Cessna Mustang (C510) [2560 flights] 

- Cirrus (SR22T) [10500 flights] 

Flood Mapping 

Total UAS Flight Hours in a Day: 25,734 
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UAS Cruise Altitudes 
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UAS Cruise Speeds 


240 - 

Mean Cruise Speed 220 
(knots) 



UAS Mission Type 
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Cooperative VFR Traffic 

The 84th Rader Evaluation Squadron (RADES) data were used. 

- The data contain the radar hits collected from hundreds of radar sites in U.S, 
and each hit provide timestamp, latitude, longitude. Mode 3 code. Mode C 
code, and others. 

- There is no explicit information that could be used to determine whether 
radar hits come from IFR flights or VFR flights. 

Criteria for filtering out VFR traffic (for each tracked flight): 

- All tracks are below 18,000 ft, 

- At least one track has the Mode 3 code of 1200, 

- Average speed ranges from 50 knots to 250 knots. 

Non-cooperative VFR radar track conning soon... 
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Method for Extracting VFR Traffic 

• Input: RADES data for a certain time period. 

• Output: Flight plan file for a fast-time simulation system, Advanced 
Concept Evaluation System (ACES). 

• Method (three steps): 

1. Generate tracks using a minimum spanning tree based clustering algorithm, 

2. Generate smooth tracks using a Kalman filter, 

3. Generate a flight plan file after reducing the number of waypoints and 
adding airports which are closest to start/end waypoints. 
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Cooperative VFR T raffic - July 25, 2013 
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ACES Simulated Traffic 


ACES simulates flight paths of UAS mission using: 

- Departure times 

- Source and destination airports 

- Flight plan (route, cruise speed and altitude) 

- "UAV-like" aircraft models 

ACES simulates flight paths of cooperative VFR traffic using: 

- Departure times 

- Source and destination airports 

- Flight plan (route, cruise speed and altitude) 

- "GA-like" aircraft model 
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Well-Clear ACES Simulations 


ACES agent-based distributed architecture 

Typical configuration is running across three servers with 36 CPU-threads 
and 360 GM RAM 

Each UAV aircraft is modeled by an ''agent'', and balanced across the 36 
CPUs 

Within the agent a DAA "activity" was added that is configured to accept 
all truth states with a "gross" filter distance at a parametric detect rate 

- 2-second detect rate 

- 30 nmi range 
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Well-Clear ACES Simulations 
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DAA Architecture (Unmitigated) 


Detect Rate 
Timer 


Encounter Metrics- 
Calculator 



Ownship State and intent data 
Intruder Surv. Data 


ACES 


Measures: 

• ModTau 

• VertTau 

• TimeToCpa 

• HMD 

• VMD 


rDot 

hDot 

Range 

reIX 

relY 

relZ 

relHeading 
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Integration of Well-Clear Definitions into Autoresolver 


• Three well-clear definitions have been integrated into Autoresolver 

• At every detect cycle, 2-minute trajectories for ownship and intruder are 
built 

- No sensor uncertainty 

- Only intruder position and velocities are known 

• If well-clear violation is predicted to occur within a parametric time (SST), 
Autoresolver is triggered to generate an avoidance maneuver 

- Due to maneuvering intruders, or bends in UAV route structure may detect 
before the SST 



If within SST: maneuver 
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Autoresolver Resolution Logic 


• Detect well-clear violations with at 60 seconds SST 

• A prominent feature of Autoresolver is during re-evaluation, trajectories 
for maneuver are built until SST plus buffer 

- E.g. detect at 60 seconds, and recommend resolution predicted to be threat- 
free for 120 seconds 

• Another prominent feature of Autoresolver is adding buffer to horizontal 
separation 

- E.g. scale horz. separation by 25% 

- Usually no need to add buffer to vertical domain, because procedural 
significance of vertical clearances (cardinal altitudes) 

• Lastly, for NASA and MIT-LL well-clear definitions. Autoresolver tries to 
maneuver to achieve well-clear via vertical or horizontal miss distance, 
modTau not considered, but TCOA is 
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Autoresolver Resolution Logic (cont.) 

Searches for the minimal left/right turn and climb/descent that resolves 
the WCV, i.e. respective WCV threshold plus a 25% scaling in horizontal 

- Integrates 5 degree turns, and 500 ft altitude increments 

Minimizes deviation, i.e. minimum cross track and vertical deviation plus 
buffers 

Resolution heuristic prefers horizontal maneuver when ownship is at level 
flight 

Prefers vertical maneuvers if the ownship is non-level 


In the event of tie in maneuver deviation, turning right or descending is 
preferred 
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Lessons Learned 


WCV with VFR traffic may end and start close to each in time 
- Merge if WCV between same aircraft pair is within 120 seconds 

Time to co-altitude's utility not realized given limited number of high 
vertical closure rate encounters 


P(TCAS-RA| WCV) is high at TCAS Sensitivity Level 6 (Alt: 10,000 - 20,000) 
for all WC definitions 

- Parameters of TCAS model thresholds are outside NASA and MIT-LL 
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TCAS RA Model Altitude Dependent Thresholds 


1 Own Altitude (ft) 

SL 

Tau 

(sec) 

DMOD 

(nmi) 

ZTHR (ft) 

ALIM 

( ft ) 

HMD (ft) 1 

1000 - 2350 

3 

15 

0.20 

600 

300 

1215 

2350 - 5000 

4 

20 

0.35 

600 

300 

2126 

5000 - 10000 

5 

25 

0.55 

600 

350 

3342 

10000 - 20000 

6 

30 

0.80 

600 

K 

400 

4861 

20000 - 42000 

7 

35 \ 

/ 1.10 

700 \ 

600 

6683 

> 42000 

7 

35 / 

V 1.10 

800 \ 

700 

6683 


MIT-LL uses 0.66 nmi 


NASA uses 475 ZTHR 


NASA uses 20 TCOA 


* Source: ”A TCAS-II RESOLUTION ADVISORY DETECTION ALGORITHM,” 
Cesar Munoz, Anthony Narkawicz, and James Chamberlain, 

AIAA Guidance, Navigation, and Control Conference, 2013. 

Table 1: TCAS Sensitivity Level Definition and Alarm Thresholds for RAs 
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Final Results 
(Unmitigated) 
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P(Corr-RA| WCV) Unmitigated ACES Result 


Analyzing P(Corr-RA | WCV) as a function of TCAS Sensitivity Level 3-6 


P(Corr-RA I WCV) 



■ ■ ■ ■ 

MIT-LL NASA USAF 


TCAS-Level within WC Def. 
TCAS-Level *3 *4 Hs H6 

3 = [1,000 - 2,350 ft] 5 = [5,000 - 10,000] 

4 = [2,350 - 5,000 ft] 6 = [10,000 - 20,000] 
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P(NMAC|WCV) 


NASA = 1.17% 


MIT-LL = 1.23% 


USAF = 1.37% 
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Ownship Altitude at NMAC 


Quantiles 


- 0.30 

- 0.25 

- 0.20 

- 0.15 

— I -0-10 



0 2000 4000 6000 8000 10000 

Ownship Altitude at WCV 
(feet) 



100-0% 

max 

10000 


99-5% 


10000 


97-5% 


10000 


90-0% 


7733-6 


75-0% 

quartile 

4416 


50-0% 

median 

3274 

o 

25-0% 

quartile 

1615 

CC 

o 

10-0% 


899 

o 

2-5% 


798-15 

CL 

0-5% 


782 


0-0% 

min 

782 


Summary Statistics 


Mean 


3564-2338 


Std Dev 


2300-1527 


Std Err Mean 

262-12673 


Upper 95% Mean 

4086-3044 


Lower 95% Mean 

3042-1631 


N 


77 
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B 


No. of WCV per Day per Definition 



WC Def. 

MIT-LL 

NASA 

USAF 
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No. of WCV by UAS Mission 


N 



WC Def. 

MIT-LL 

NASA 

USAF 


MissionType 
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Encounter Angle at WCV 


MIT-LL 


WC Def. 

NASA 


Encounter Angle 

^■Crossing 

Head-on 

USAF ■ In-trail 



Bearing 


In-Trail: 0-45 deg. 
Crossing: 45-135 deg. 
Head-On: 135-180 deg. 
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Horz. And Vert. Range at WCV - NASA 

1600 
1200 
800 

Vert Range 400 
At WCV (feet) 

0 

-400 
-800 
-1200 

0 5000 10000 15000 20000 25000 30000 

Horz Range At WCV (feet) 

• Larger vertical range is due to time to co-altitude. 

• High vertical range and low horz. range may be difficult for RADAR to detect 
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Horz. And Vert. Range at WCV - MIT-LL 


1600 - 


Vert Range 
At WCV (feet) 


1200H 
800 
400 


0 - 


-400 

-800 

-1200 





0 


5000 


10000 15000 20000 

Horz Range At WCV (feet) 


T 

25000 


T 

30000 


• Larger horz range is expected due to larger modTau 

• Lack of TCOA threshold makes vertical range truncated at 700 ft 
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Horz. And Vert. Range at WCV - USAF 

1600 
1200 
800 

Vert Range 
At WCV (feet) ^ 

-400 
-800 
-1200 

0 5000 10000 15000 20000 25000 30000 

Horz Range At WCV (feet) 

• Much shorter ranges, positive for sensor requirements 
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Horizontal Range at WCV 


Horz Range 
(feet) 


WC Def. 

MIT-LL NASA USAF 
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Vertical Range at WCV 
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ModTau/Time-to-co-altitude Model 


Question: How often does time to co-altitude get triggered with NASA WCVs? 


[ ( r <= DMOD ) or 

((0 <= tau_mod <= tau_thresh) and 
(HMD <= HMD_thresh) ) ] 

and 


[(h <= ZTHR) or 

(0 <= vert_tau <= tau_thresh)] 
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NASA Definition Violation Probability 


N(WCV) 

P(ModTau_TCOA) 

P(DMOD_TCOA) 

P{ModTau_ZThr) 

P(DMOD_ZThr) 

6473 

1 . 3 % 

1 . 2 % 

80 . 8 % 

16 . 7 % 


Key: 


ModTau_TCOA: 
DMOD_TCOA: 
ModTau_ZThr; 
DMOD ZThr; 


WCV occurs with modTau and TCOA 

WCV occurs with DMOD and TCOA 

WCV occurs with modTau and ZThr (not TCOA) 

WCV occurs with DMOD and ZThr (not ModTau or TCOA) 
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hDot (vertical closure rate) - NASA 


Quantiles 



hDot (fpm) 


100 - 0 % 

99.5% 

97.5% 

90.0% 

75.0% 

50.0% 

25.0% 

10 . 0 % 

2.5% 

0.5% 

0 . 0 % 


max 


quartile 

median 

quartile 


mm 


6665 

2359.94 

1465.3 

609.601 

46.9998 

-151 

-985 

-1562.6 

-2377.6 

-3174.8 

-5268 


Summary Statistics 


Mean 

Std Dev 

Std Err Mean 

Upper 95% Mean 

Lower 95% Mean 

N 


-375.2541 

948.14354 

1 1 .784775 

-352.1521 

-398.3562 

6473 
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hDot (vertical closure rate) - MIT-LL 


Quantiles 



7500 -5500 -3500 -1500 0 1000 2500 4000 

hDot (fpm) 




100.0% 

max 

4308 



99.5% 


2455.86 



97.5% 


1570.62 

- 0.20 


90.0% 


741.701 

>> 

75.0% 

quartile 

99 

- 0.15 

H— ' 

'o 

50.0% 

median 

-31 


cc 

25.0% 

quartile 

-744 

- 0.10 

_LJ 

o 

10.0% 


-1438.7 


CL 

2.5% 


-2194.5 

- 0.05 


0.5% 


-3023.7 



0.0% 

min 

-6730 



Summary Statistics 



Mean 


-246.0974 



Std Dev 


909.73485 



Std Err Mean 

1 1 .742669 



Upper 95% Mean 

-223.0776 



Lower 95% Mean 

-269.1173 



N 


6002 
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hDot (vertical closure rate) - USAF 


Quantiles 



5500-4000 -2500 -1000 0 1000 2500 4000 

hDot (fpm) 




100-0% 

max 

4309 



99-5% 


2365-84 



97-5% 


1647-35 

- 0.20 


90-0% 


737-699 


>> 

75-0% 

quartile 

91-998 

- 0.15 

H— ' 

50-0% 

median 

-47 

CC 

o 

25-0% 

quartile 

-756-75 

- 0.10 

O 

10-0% 


-1442 


CL 

2-5% 


-2062-2 

- 0.05 


0-5% 


-3095-9 



0-0% 

min 

-5199 



Summary Statistics 



Mean 


-251-3393 



Std Dev 


910-31886 



Std Err Mean 

12-13004 



Upper 95% Mean 

-227-5598 



Lower 95% Mean 

-275-1189 



N 


5632 
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IFR-VFR 500ft Altitude Separation - Well-Clear? 

• MIT-LL uses 700 feet ZTHR 

• NASA uses 475 feet ZTH R 

Question: If two aircraft pass at 500ft and at level flight isn't that well-clear? 


TCAS 

Ownship 

Percentage of WCVs that 

Sensitivity 

Altitude (ft) 

pass above 500ft 


3 

1,000-2,350 

2.4% 

4 

2,350-5,000 

4.1% 

5 

5,000-1,0000 

3.4% 

6 

10,000-20,000 

0.5% 

Total 

- 

10.5% 


Magnitude of MIT-LL that pass above 500ft and vertical closure rate is iess than 100 FPM (ievei). 




Overview of ACES Simulation for Evaluating SARP Well-Clear Definitions 


37 





WCV Duration - NASA 


Quantiles 



JD 

O 


1 00a0% 

99-5% 

97,5% 

90.0% 

75.0% 

50.0% 

25.0% 

10 . 0 % 

2.5% 

0.5% 

0 . 0 % 


max 


quartile 

median 

quartile 


mm 


314 

169.26 

100 

66 

50 

36 

18 

8 

2 

2 

2 


Summary Statistics 


WCV Duration 
(sec) 


Mean 

Std Dev 

Std Err Mean 

Upper 95% Mean 

Lower 95% Mean 

N 


38.042021 

27.380679 

0.3403231 

38.709166 

37.374875 

6473 
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WCV Duration -MIT-LL 


Quantiles 



JD 

CO 

JD 

O 


100.0% max 552 

99.5% 141.97 

97.5% 87.85 

90.0% 58 

75.0% quartile 48 

50.0% median 38 

25.0% quartile 18 

10 . 0 % 8 

2.5% 2 

0.5% 2 

0.0% min 2 

Summary Statistics 


(sec) 


Mean 

Std Dev 

Std Err Mean 

Upper 95% Mean 

Lower 95% Mean 

N 


36.176608 

24.139063 

0.311582 

36.787421 

35.565795 

6002 
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WCV Duration - USAF 


Quantiles 



0 20 60 100 140 180 220 260 300 

WCV Duration 




100.0% 

max 

298 



99.5% 


131.67 



97.5% 


76.35 

0.25 


90.0% 


44 

0.20 


75.0% 

quartile 

32 


50.0% 

median 

20 

0.15 

CO 

25.0% 

quartile 

10 

0.10 

o 

Q_ 

10.0% 


4 



2.5% 


2 

0.05 


0.5% 


2 



0.0% 

min 

2 


Summary Statistics 


(sec) 


Mean 

Std Dev 

Std Err Mean 

Upper 95% Mean 

Lower 95% Mean 

N 


23.896662 

20.776644 

0.2768497 

24.439394 

23.35393 

5632 
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Ownship Altitude at WCV - NASA 


Quantiles 



0 2 4 6 8 10 12 14 16 18 

Ownship Altitude at WCV 
(10^* feet) 




100-0% 

max 

17 



99.5% 


10.8118 



97.5% 


10 

- 0.25 


90.0% 


8 

- 0.20 


75.0% 

50.0% 

quartile 

median 

5 

3.44 

- 0.15 

CO 

25.0% 

quartile 

2.1085 


_Q 

O 

10.0% 


1.2882 

- 0.10 

CL 

2.5% 


0.82985 

- 0.05 


0.5% 


0.759 


0.0% 

min 

0.75 



Summary Statistics 



Mean 


3.9361542 



Std Dev 


2.3863824 



Std Err Mean 

0.0296611 



Upper 95% Mean 

3.9942997 



Lower 95% Mean 

3.8780086 



N 


6473 
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Ownship Altitude at WCV - MIT-LL 


Quantiles 




100.0% 

max 

16.191 



99.5% 


10.5522 



97.5% 


10 

- 0.25 


90.0% 


8 

- 0.20 


75.0% 

quartile 

5 


50.0% 

median 

3.471 

- 0.15 

CO 

25.0% 

quartile 

2.14875 

_LJ 

o 

10.0% 


1.321 

- 0.10 

Q. 

2.5% 


0.83 

- 0.05 


0.5% 


0.761 


0.0% 

min 

0.752 


0 


8 10 12 14 16 


Summary Statistics 


Ownship Altitude at WCV 
(10^* feet) 


Mean 

Std Dev 

Std Err Mean 

Upper 95% Mean 

Lower 95% Mean 

N 


3.9732962 

2.407336 

0.0310734 

4.0342113 

3.9123812 

6002 
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Ownship Altitude at WCV - USAF 


Quantiles 



0 2 4 6 8 10 12 14 16 18 

Ownship Altitude at WCV 
(10^* feet) 




100-0% 

max 17 



99-5% 

1 1 -7964 

-0.25 


97-5% 

10 



90-0% 

8 

-0.20 


75-0% 

quartile 5-12825 

-0.15 


50-0% 

median 3-464 

CO 

c 

25-0% 

quartile 2-097 

-0.10 

0 

Q 

10-0% 

1-291 


2-5% 

0-828 

-0.05 


0-5% 

0-76017 



0-0% 

min 0-752 



Summary Statistics 



Mean 

4-0715243 



Std Dev 

2-5496832 



Std Err Mean 0-0339746 



Upper 95% Mean 4-1381277 
Lower 95% Mean 4-0049209 



N 

5632 
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Conclusion 

Tomorrow I will dig deeper in results for the SARP accepted metrics 

P(NMAC| WCV) is in the ball-park of the tuned 1.5% 

For all UAS mission types each WC definition has similar performance 

TCAS Sensitivity level between 10,000-20,000 feet shows high P(TCAS-RA 
WCV) for all WC definitions 

Doesn't look like TCOA really captures many encounters given LL 
encounter model and ACES UAS vs. VFR encounters 
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ACES Unmitigated fand Some Mitigated! Results 
Supporting Selection of SARP Well-Clear 

Definition 

Confesor Santiago 
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Hershev 


TCAS Model Summary 

As a proxy for whether a TCAS Corrective RA would be presented we use 
model published by NASA Langley in GNC 2013 paper 

Given UAV encounter, TCAS RA model is computed from standpoint of the 
intruder (assumed TCAS equipped) 

At every cycle as intruder encounters UAV, we compute if Equation (12) 
and Equation (14) are true, then its marked as a TCAS RA 

Mathematical model is the same as the one used by NASA and MIT-LL for 
well-clear definition (modTau and time to co-alt) 

However, altitude dependent thresholds are used based on intruder's 
own altitude 

Also, there is a single tau threshold (for SARP we decoupled modTau and 
time-to-co-altitude) 
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TCAS RA Model Altitude Dependent Thresholds 


1 Own Altitude (ft) 

SL 

Tau 

(sec) 

DMOD 

(nmi) 

ZTHR (ft) 

ALIM 

( ft ) 

HMD (ft) I 

1000 - 2350 

3 

15 

0.20 

600 

300 

1215 

2350 - 5000 

4 

20 

0.35 

600 

300 

2126 

5000 - 10000 

5 

25 

0.55 

600 

350 

3342 

10000 - 20000 

6 

30 

0.80 

600 

400 

4861 

20000 - 42000 

7 

35 

1.10 

700 

600 

6683 

> 42000 

7 

35 

1.10 

800 

700 

6683 


Intruder Altitude: 2.000 ft - 17.999ft 

* Source: ”A TCAS-U RESOLUTION ADVISORY DETECTION ALGORITHM,” 
Cesar Munoz, Anthony Narkawicz, and James Chamberlain, 

AIAA Guidance, Navigation, and Control Conference, 2013. 

Table 1: TCAS Sensitivity Level Definition and Alarm Thresholds for RAs 
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TCAS RA Model 


[ ( r <= DMOD ) or 

((0 <= tau_mod <= tau_thresh) and 
(HMD <= HMD_thresh) ) ] 

and 


[(h <= ZTHR) or 

(0 <= vert_tau <= tau_thresh)] 


Preventive: vertDistCPA >= ALIM 

Corrective: vertDistCPA < ALIM 
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P(TCAS-RA I WCV) Unmitigated ACES Result 

Probability of Well-Clear Violation (WCV) with TCAS RA prior to WCV 

Assumption: Intruders (manned) experiencing TCAS-RA's while UAS DAA 
system detects it as well-clear is undesirable. 

The smaller the better 

Number of WCVs with TCAS-RA prior to WCV 
Totai Number of WCVs 

To measure TCAS RA used data from 2 seconds prior to WCV 
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TCAS-RA Rates Mitigated Result 


• While detecting and resolving for WCVs (mitigated), at what rate do we 
trigger a TCAS-RA? 

- Didn't have time for break-out of Corr-RA and Prev-RA 


# of TCAS-RA’s 
Total UAS Flight Hour 
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P(TCAS-RA I WCV) Unmitigated ACES Result 


MIT-LL 

50 

27 

77 

NASA 

224 

114 

338 

USAF 

551 

308 

839 


Total TCAS-RA Counts for all 8 days 

simulated 
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P(TCAS-RA I WCV) Unmitigated ACES Resuit 
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P(Corr-RA| WCV) Unmitigated ACES Result 
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P(TCAS-RA I WCV) Unmitigated ACES Result 


Analyzing P(TCAS-RA | WCV) as a function of TCAS Sensitivity Level 3-6 


P(TCAS-RA I WCV) 


70% 
60% H 


Undesirable 



MIT-LL NASA 

TCAS-Level within WC Def. 


USAF 


Desirable 


TCAS-Level 

3 = [1,000 - 2,350 ft] 5 = [5,000 - 10,000] 

4 = [2,350 - 5,000 ft] 6 = [10,000 - 20,000] 
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P(Corr-RA| WCV) Unmitigated ACES Result 


Analyzing P(Corr-RA | WCV) as a function of TCAS Sensitivity Level 3-6 


P(Corr-RA I WCV) 



■ ■ ■ ■ 

MIT-LL NASA USAF 


TCAS-Level within WC Def. 
TCAS-Level *3 *4 Hs H6 

3 = [1,000 - 2,350 ft] 5 = [5,000 - 10,000] 

4 = [2,350 - 5,000 ft] 6 = [10,000 - 20,000] 
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P(TCAS-RA I WCV) Unmitigated ACES Resuit 


P(TCAS-RA I WCV) 



MIT-LL NASA USAF 



Prob. Corr-RA 
Prob. Prev-RA 
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Illustration of Typical USAF Case with TCAS-RA 



Lon 


• A lot of the time, TCAS-RA model, which is based on modTau, triggers farther out 
than the length of the horizontal cone for head-on and crossing encounters 

• More of an issue in the modTau, rather than the Z_THR 
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NASA vs. MIT-LL Unmitigated TCAS-RA Comparison 


• Identified 266 NASA WCVs where: 

- An associated MIT-LL WCV occurred with same encounter pair 

- Altitudes at WCV are within 300 feet of MIT-LL 

- TCAS-RA occurred prior to time of NASA WCV 

- TCAS-RA occurred after MIT-LL WCVs 

• These cases are the majority of why P(TCAS-RA | WCV) is higher for NASA 
WC definition. 



0 10 20 30 40 50 60 70 80 

Distribution of (NASA WCV Time - MIT-LL WCV Time) 

seconds 


Mean 

14.9 

Std Dev 

9.8 

Median 

10 

Min 

2 

Max 

76 

N 

266 
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NASA vs. MIT-LL Unmitigated TCAS-RA Comparison 


0 250 500 750 1000 1250 1500 1750 2000 

Distribution hDot 
(FPM) 


Mean 

1084.8 

Std Dev 

469.5 

Median 

1253 

Min 

0 

Max 

1873 

N 

266 


• At the lower vertical closure rates (hDot), MIT-LL's 700 feet ZTHR protects against TCAS-RA at 
600 feet 

• 97.5% of the WCVs occur below 10,000ft; TCAS uses at 25 tau threshold there 

• In case of larger vertical closure rates, TCOA of 20 is within TCAS limits, hence TCAS-RA 
occurring prior to WCV 

• MIT-LL's 700 feet Z_THR equates to ~33.5 seconds TCOA for median hDot = 1253 fpm, which 
is outside TCAS's 25 tau threshold, and ~29 seconds TCOA for TCAS-RA model 
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P(TCAS-RA I WCV) Unmitigated ACES Result 


Final Result 
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TCAS-RA Rate Mitigated 

• NASA: 3.9 x 10"^ RA/fIt-hour (filtered out anomaly WCVs) 

• MIT-LL: 8.47 x 10'^ RA/fIt-hour (filtered out anomaly WCVs) 

• USAF: 1.52 x 10"^ RA/fIt-hour (filtered out anomaly WCVs) 
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WCV Rate Unmitigated and WCV/Encounter 

Rate Mitigated ACES Result 
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WCV Rate Unmitigated ACES Result 

Well-Clear Volume Penetration Rate per Flight Hour 

Intuition tells me the lower the rate the better. 

The complement is interesting, because the higher the rate points to 
presumably that the well-clear definition is larger, which may make the 
system safe, however may have negative affect on interoperability to ATC. 

There is a tradeoff here... 

Number of SST Encounters 

Number of WCVs where Maneuver is Required 

Number of flight hours Number of flight hours 


Overview of ACES Simulation for Evaluating SARP Well-Clear Definitions 


WCV Rate Unmitigated ACES Result 


Well-Clear Violation 
Rate per 

1 X 10^ flight hours 



MIT-LL 


NASA USAF 

Bearing within WC Def. 


Bearing 


Crossing 


Head-on 


In-trail 
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WCV Rate Unmitigated ACES Result 



MIT-LL NASA USAF 


TCAS-RA_Alt within WC Def. 
TCAS-RA_Alt *3 *4 HS H6 

3 = [1,000 - 2,350 ft] 5 = [5,000 - 10,000 ft] 

4 = [2,350 - 5,000 ft] 6 = [10,000 - 20,000 ft] 
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WCV Rate Unmitigated ACES Result 



WC Def. 

MIT-LL 

NASA 

USAF 


WCV Rate reveals similar trends within each day for each WC definition. 
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WCV Rate Unmitigated ACES Result 
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WCV Rate reveals similar trends within each day for each WC definition. 
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WCV Rate Unmitigated ACES Result 



MissionType 
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WCV Rate Unmitigated ACES Result 


Flight Hours 




Flight hours per mission type within each day simulated 
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WCV Rate Unmitigated ACES Result 


Final Result 

3.5 

3.0 

2.5 

Well-Clear Violation 
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1 X 10^ flight hours 
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WCV Rate reveals similar trends within each day for each WC definition. 



MIT-LL NASA USAF 
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WCV Rate Mitigated ACES Resuit 


Final Result 


NASA 1.97x103 1.30x101 

MIT-LL 1.55 X 103 ^ q5 x 10^ 

USAF 5.60x 103 1.76x 101 


* SST_rate is the rate in which mitigation was needed, i.e. crossing SST 


• For all WC definitions mitigation was able to reduce WCV rate by an order 
of magnitude (100s of fit-hours to 1000s fit-hours) 

• Similar to WCV rate in unmitigated, the rate of ''requiring'' self-separation 
maneuver is about the same 

• NASA and MIT-LL in ballpark, USAF is a couple multiples larger 
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Miss Distances at CPA 

and 

Minimum Time Until NMAC from WCV 


Unmitigated ACES Result 
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Miss Distances at CPA 


Intuition tells me the higher the miss distances the better. 

Although, based on unmitigated encounters only, miss distances may not 
be a good indicator of WC definition performance. 

Two definitions of CPA: 

- Minimum slant range between ownship and intruder 

- Minimum slant range normalized for NMAC 500ft hSep and 100ft vSep (5-to-l 
ratio) 
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Minimum Time Until NMAC from WCV 


Intuition tells me the higher the minimum time until NMAC the better. 

In other words, the more time between becoming a WCV to becoming 
NMAC the "safer". 

Since the unmitigated P(NMAC | WCV) and WCV rate is about the same, 
this is a way to analyze that not every WCV is created equal. Which 
definition is generally closer to an NMAC. 

Not only is minimum shown, but also the distribution. 

Min (NMAC_time - WCV_time) 
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Predicted Min. Slant Range CPA - Miss Distances 


Horz. Miss Distance (ft) 
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• Miss distances for NASA and MIT-LL perform as expected given their 
specific DMOD/HMD 

• USAF without a miss distance filter at CPA results in larger HMDs 
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Min. Weighted Slant Range CPA - Miss Distances 


Horz. Miss Distance (ft) 



MIT-LL NASA USAF 
WC Def. 


Vert. Miss Distance (ft) 



WC Def. 


• Miss distances for NASA and MIT-LL perform as expected given their 
specific DMOD/HMD 

• USAF without a miss distance filter at CPA results in larger HMDs 
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Minimum Time Until NMAC from WCV (sec) 



Stats 



maximum 
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NASA Min. Time Untii NMAC from WCV 


• NASA WCV case for minimum time until NMAC, and how it compares to 
similar case for other WC definitions. 


NASA 

16 

11,222 

476 

11.7 

18.2 

146.4 

-241 

-1,572 

3,036 

2,560 

MIT-LL 

24 

16,715 

686 

22.9 

26.1 

146.4 

-241 

-1,572 

3,245 

2,560 

USAF 

14 

9,849 

424 

- 

_ 

146.4 

-241 

-1,572 

2,984 

2,560 


Take-away: 

• Given -1,572 FPM vertical closure rate the MIT-LL WC definition is 
triggered 8 seconds prior to NASA's, because when violating the TCOA is 
above 20 seconds (26.1 seconds). 

• USAF time to NMAC is later due to smaller vSep requirement 
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MIT-LL Min. Time Until NMAC from WCV 


• MIT-LL WCV case for minimum time until NMAC, and how it compares to 
similar case for other WC definitions. 


NASA 

22 

4,600 

842 

0 

18.9 

23.7 

-74.0 

-2,673 

4,858 

4,016 

MIT-LL 

18 

3,779 

666 

0 

15.2 

23.6 

-72.9 

-2,616 

4,734 

4,069 

USAF 

12 

2,250 

408 

_ 

_ 

23.5 

-71.6 

-2,568 

4,549 

4,141 


Take-away: 

• Given -2,673 FPM vertical closure rate the NASA definition is triggered 4 
seconds prior to NASA's, because when violating the TCOA is below 20 
seconds (15.2 seconds). 

• USAF time to NMAC is later due to smaller vSep requirement, and large 
vertical closure rate 
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USAF Min. Time Untii NMAC from WCV 


• USAF WCV case for minimum time until NMAC, and how it compares to 
similar case for other WC definitions. 


NASA 

36 

12,030 

337 

27.7 

0 

34.9 

-114.2 

90.9 

1,957 

1,619 

MIT-LL 

36 

12,030 

337 

32.8 

0 

34.9 

-114.2 

90.9 

1,957 

1,619 

USAF 

8 

2,916 

122 

_ 

_ 

35.8 

-112.5 

-345.6 

1,851 

1,729 


Take-away: 

• NASA and MIT-LL triggered much earlier than (24 sec) USAF due to modTau 

• Due to smaller vSep requirement of USAF, and the fact that the ownship 
and intruder suddenly accelerate towards each in vertical closure rate, time 
to NMAC is short 

- Note, when NASA and MIT-LL WCV triggered, vertical closure is diverging 
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Minimum Time Until NMAC from WCV 


Final Result 


NASA 

16 

MIT-LL 

18 

USAF 

8 
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